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A spectral study of power-map deformed singular random correlation matrices is presented in 
this paper. The study is inspired from the multivariate analysis of large dimensional datasets where 
stationarity or quasi-stationarity could often be a good approximation for a non-stationary time 
series only over a short time horizon. Correlation matrices estimated over such short time horizons 
are singular, with a significant portion of the spectrum falling exactly on zero. The power map 
has been proposed to remove this spectral degeneracy, where the correlation matrix is deformed as 
soon as the exponent is varied from one. Importance of the power map has been illustrated for 
the singular matrices encountered in financial application, viz. the portfolio optimization. To get 
a clearer picture of deformed matrices, the exponent of the power map has been chosen to be very 
close to one such that the emerging spectrum corresponding to the zeros of the original spectrum 
falls into a gap between zero and nonzero part of the entire spectrum. Analytical approximate 
estimate of the first and the second moments for the emerging spectra have been calculated for the 
deformed Wishart matrices of random matrix theory. Furthermore, the study has been extended 
to the correlated Wishart ensembles with two examples, viz. when the true correlation matrix has 
block-diagonal structure while in the second example it has a smooth band profile off the diagonal. 
In both cases the spectra have been analyzed numerically using the analytical results as a benchmark 
situation. It is shown that the emerging spectra are sensitive to the true correlation. 

PACS numbers: 02.50.Sk, 05.45.Tp, 89.90.Gh, 89.65.+n 
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I. INTRODUCTION 

Physics has long concentrated on equilibrium states, 
but more recently stationary states have drawn increas- 
ing attention, particularly in all kinds of transport rang- 
ing from heat over charge and partcle transport all the 
way to traffic models [lj-ij] . Yet it has become very clear 
that stationarity is at best an approximation for fairly 
short times in many applications. In this context analysis 
of financial markets have been in the forefront 0-Q > an d 
today the new frontier is the analysis of non-stationary 
systems with a hope that it can be extrapolated from 
quasi-stationarity over shorter time periods. A nice ex- 
ample thereof is a recent paper on market states [lOJ . 

Starting with early work in biology, multivariate anal- 
ysis [ll|, [l2| has been established as an essential tool. 
In particular, correlation matrices play an important 
role in many problems of biology (l3l . Iljj . medical re- 
search [15j, econophysics [la - ibSl ]. etc. The eigenlevel- 
eigenvector analysis of such positive definite matrices has 
been remarkably useful in the time series analysis of mul- 
tidimensional complex systems [19I426J . Usually in such 
studies the time horizon is considered to be longer than 
the dimensionality of datasets to get rid of spectral re- 
dundancies. 
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Random matrices were introduced by Wishart [27j as 
a fundamental tool in multivariate analysis. After their 
successful applications in nuclear and atomic physics 
[28l l29j random matrices achieved considerable atten- 
tion form physicists which resulted in the foundation of 
random matrix theory (RMT). Consequently, RMT has 
been extensively used and developed in diverse areas of 
research, for example, in physics |30ll31|. in communica- 
tion engineering [32[ and in econophysics J33j]. Wishart 
introduced random matrices of type C = AA [T where 
the matrix A is N x T, A* is the transpose of A and 
matrix elements Ajj. are independent Gaussian variables 
with mean zero and variance one. In this model C char- 
acterizes a null hypothesis where the covariance matrix is 
calculated, over stochastic time series of time horizon T, 
for N statistically independent variables. In random ma- 
trix theory (RMT) ensembles of such matrices are well 
studied and known as Wishart or Laguerre ensembles 
(WE) [28]. Analogous to WE the correlated Wishart 
ensembles (CWE) are defined where matrix elements of 
A can have genuine nonrandom 'true correlations' [111 ]. 
Recently, important advances have been made for CWE 
|34J4fj| which go beyond the null hypothesis and set a 
theoretical background for incorporating the true corre- 
lations. Ongoing research activities on time series analy- 
sis of complex systems often exploit results of such ran- 
dom matrix models, mainly because these models supply 
a framework to filter signals from noise or for quantify- 
ing the noise [19M24L I26J. While the eigenlevel statistics, 
e.g. density of eigenlevels, have been useful for denoising 



the spectrum, the eigenvector analysis has been useful 
in identifying the collective movements in the datasets 

MM. 



For short time series, such as we necessarily obtain 
for non-stationary or quasi-stationary situations, often 
the time horizon is smaller than the dimensionality of 
datasets to be analyzed. In that case, because of the 
factorisable structure, C will be a singular correlation 
matrix with at least N — T zero eigenlevels. The cor- 
responding random matrix ensemble is sometimes called 
" Anti- Wishart ensemble" [4j], |42| ■ For such matrices an 
analysis of the eigenvectors can only be fruitful for the 
nonzero eigenlevels, as those of the zero eigenlevels are 
only defined up to an orthogonal transformation in the 
degenerate subspace. However, we propose to remove 
such degeneracy by using the so-called power map which 
has been successful for regular, or full rank, correlation 
matrices encountered in financial applications 43-45]. 
Interestingly, this method produces significantly useful 
results for singular C, as well, as we show in this paper. 
In the present paper we are not so much interested in 
the noise reduction aspect of the power map, but in the 
breaking of the degeneracy. We will get a much clearer 
picture by using the exponent values sufficiently close to 
one so that the part of the spectrum originating from 
the originally-zero eigenlevels will fall entirely into the 
gap between zero and the bulk of the spectrum which is 
close to the spectrum of the correlation matrix. We thus 
obtain a disjoint spectrum of eigenlevels which we shall 
call the emerging spectrum. 

We derive analytically approximate estimate for the 
first two moments, for the density of emerging eigenlevels 
and also for the density of the corrections to non-zero 
eigenlevels of C, in a linear response regime. In analyti- 
cal work we mostly consider WE because WE is not only 
analytically more tractable than CWE but it also belongs 
to a benchmark situation or the null hypothesis, as men- 
tioned above. Besides, we study spectral statistics for 
power map deformed correlation matrices corresponding 
to CWE. We show that the true cross-correlations change 
the spectral statistics appreciably. For CWE we consider 
two types of ensemble or true correlation matrices. The 
first one correspond to the 'factor models' which have 
been useful in quantitative finance 
has a smooth band profile. 



and the other one 



The paper is organized as follows. In Sec. II we il- 
lustrate the power map for singular correlation matrices 
and describe its importance for the portfolio optimiza- 
tion. In Sec. Ill we summarize useful results known for 
WE and for CWE, specially those which we use in this 
paper. In Sec. IV, we derive estimates of the first and the 
second moment of the density of small eigenlevels of the 
power mapped Wishart matrices. In Sec. V we consider 
non-vanishing true cross correlation in C, for CWE, and 
analyze the spectra of power mapped matrices. This is 
followed by conclusion in Sec. VI. 




FIG. 1: Portfolio variance Q, 2 normalized to the minimal port- 
folio variance Q, 2 ,. The length T of the time series was varied, 
the number of companies was fixed at N = 100. Results are 
shown for sample correlations without noise reduction (black 
dashed-dotted line) and for power-mapped correlations (blue 
solid line). The red dashed line at Q. 2 /Q. 2 = 4.37 corresponds 
to a homogeneous portfolio with all weights equal to 1/N. 



II. POWER MAP FOR SINGULAR 
CORRELATION MATRICES 

We wish to briefly illustrate the power map for 
singular correlation matrices. For instance, following 
Markowitz [16[ , we consider a portfolio of N stocks and 
wish to calculate the portfolio weights u> op t which mini- 
mize the portfolio variance 



& = w opt T,w opt . 



(1) 



In a model setting, we can calculate the minimal vari- 
ance portfolio S7q using the model covariance matrix So- 
In practice, however, the covariance matrix has to be 
estimated using historical data of finite length T. The 
shorter the length T of the time series, the noisier is the 
covariance estimation. Using noisy covariance matrices 
for portfolio optimization leads to very bad results, see 
Fig. [TJ In this case, the portfolio variance f2 2 increases as 
(1 — N/T)~ x , in accordance with literature J47|. Clearly, 
it is necessary to improve the estimation of the covari- 
ance matrix to obtain better results. The variances of 
the single stocks can be estimated rather well on short 
time horizons due to a slowly decaying autocorrelation. 
The noise in the correlation coefficients Cki can be re- 
duced effectively using the power map 



C, 



(?) 

kl 



sign C k i Cm 



(2) 



The results are presented in Fig. [I] The power map yields 
portfolio variances which are well below the homogeneous 
portfolio, even for N > T where the correlation matrix 
becomes singular. The values for q used in this study 
range from 1.1 to 2.4. For details of the simulation we 
refer to Appendix lAl 
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FIG. 2: Density of eigenlevels of C' 9 ' where q = 1.001, 
AT = 1024 and k = 1/2. In Fig. |^a) we show the density 
of emerging spectra while in Fig. HJb) we show the density 
of the non-zero eigenlevels which is closely described by the 
Marcenko Pastur law (Jfjl . Both densities are shown on differ- 
ent scales. Density in Fig. [2ja) is normalized to 1 — k while 
density in Fig. O^b) is normalized to «. Note that the den- 
sity in Fig. [2ja) is not symmetric, as it gives an impression 
of some kind of semi-circle. The problem dealt in this paper 
addresses to the moments of the density in Fig. [2ja). 



As far as spectral statistics is concerned, we note that 
Q( q ' is always Hermitian, but for q ^ 1 it is not necessar- 
ily a positive definite matrix in general. For short time 
horizons, when q is close to 1, the density of the eigen- 
levels of C( g ) is described on two cuts, one which is close 
to and the other is close to the cut which describes 
the density of the eigenlevels of C. The former comes 
because of the spontaneous breaking in the degeneracies 
of the zero eigenlevels of C. For example, for WE when 
q = 1.001, we show in Fig. |2Ja) the density of the emerg- 
ing spectra while in Fig. EJb) we show the density of the 
eigenlevels falling near to the non-zero eigenlevels of C. 
As shown in this figure, the latter is closely described by 
theory of WE which is known as the Marcenko Pastur 
law (|6]) and is represented by solid a line in the figure. 
These two cuts merge eventually to a single cut, as the 
power of q is raised. In this paper we address to the sta- 
tistical properties of the density shown in Fig. [2ja), viz. 
we derive an estimate of the first two moments of the 
density. 



III. RANDOM CORRELATION MATRICES: 
WISHART AND CORRELATED WISHART 

ENSEMBLES 

In RMT one defines basically three invariant Wishart 
ensembles [28j |. In this paper we consider only Wishart 
orthogonal ensemble (WOE). Similarly we consider 
CWOE among the three invariant CWEs. For WOE 
the matrix elements of A are real independent Gaussian 
variable with mean zero and a fixed variance a 1 . By con- 
struction C is a real symmetric semi-positive definite ma- 



trix. Using the Gaussian probability measure, the joint 
probability density (jpd) of the matrix elements of A can 
be written as 



P(A) dA oc exp 



-Tr- 



AA f 
'2^2 



dA. 



(3) 



where dA is infinitesimal volume in N x T matrix element 
space. For T > N, jpd of the matrix elements of C is 
defined in N(N + l)/2-dimensional matrix element space 
Gl|23, as 



P(C) dC ex (detC) 



[iV(«-l)-l]/2 



exp 



T 
2^2 



■TrC 



dC, (4) 



where k = T/N and dC — Yij>k dCjk Yij=i dCjj is in- 
finitesimal volume in the matrix element space. The jpd 
of eigenlevels of C, Xj, for j — 1, ..., N, may be obtained 
by transforming the variables to eigenlevel-eigenvector 
space and then integrating over the eigenvectors. It even- 
tually leads to 



iY 



P(Ai,...,Ajv) oc JJ w(Aj 



N 

u 

j>k 



Aj — At I 



(5) 



where w{\) = A^-^-^^cxphA^A^cr 2 ] i s the 
weight function of the associated Laguerre polynomi- 
als. We use bar to denote ensemble averages as in the 
above equation, while for spectral averages we use an- 
gular brackets, e.g. (•) = {tr.}/N. For k < 1, jpd of 
the matrix elements and the jpd of the eigenlevels have 
been derived in |4lL |42|, where again Laguerre weight 
functions describe jpd of the latter. Eigenvalue statistics 
of the WE are known in great detail [2g|. The global 
statistics, i.e. the eigenlevel density, as well as the local 
statistics, i.e. the n-point correlation functions for n > 2, 
of the eigenlevels are known in terms of Laguerre poly- 
nomials. However, for N, T — > 00 such that k > 1 and 
finite, the eigenlevel density converges to the Marcenko 
Pastur law 14811: 



p(X) = k 



y/(A + -A)(A-A_) 

27TCr 2 A 



(6) 



where A± = ct 2 (k _1 ' 2 ± l) 2 are the end points of the 
density. Note that for the singular case, i.e. k < 1, p(X) 
in the above equation is normalized to k and not to 1. 
Therefore, taking into account of (N — T) zeros, for n < 1 
we write 



?(A) 



y/(A + -A)(A-A_) 

27R7 2 A 



+ (1-«)*(A). (7) 



For large matrices, higher order spectral correlations or 
the n-point correlation functions, for n > 2, are the same 
as those for Gaussian orthogonal ensemble (GOE) |28j . 
therefore WE spectra have universal spectral fluctuations 
49]. 

In the case of true correlations among the matrix ele- 
ments Ajk, one defines CWE [11(. For instance, in order 



to take account of the correlation in the rows of A ma- 
trix one defines C = ^ 1 / 2 BB'^ 1 / 2 where the matrix £ is 
fixed positive definite matrix which takes account of true 
correlations and matrix elements of B are independent 
Gaussian variables with zero mean and a fixed variance, 
just as Ajk defined for the WE. We remark that in this 
definition C = er 2 £ while A T A/N is an identity matrix 
of dimension T x T. Analytical treatment to CWE [34 ? 
- l37j | is much more difficult and involved than to WE. 
However, some results are known in this case as well. 
For instance, analogous to the Marcenko Pastur result, 
resolvent or Stieltjes transform of the density is known 
as 



G{z) = 



1 



(k-1 + zG(z))Z* 



(8) 



The density p(A) can be determined uniquely via inverse 
transform 



p(A)=T-9fG(A±ie), 

7T 



(9) 



for positive infinitesimal e. For finite N and T the re- 
sult for the density has been recently obtained [? ] by 
using the supersymmetric method. For higher order spec- 
tral correlations, only asymptotic result for the two-point 
function is known [37|. Spectral fluctuations of CWE 
are not always universal rather there are deviations in 
some cases, as noted for the number variance [29|], S 2 (r), 
for large spectral correlation length r. Besides, for some 
models of the true correlation matrix £ few eigenlevels 
are found to be separated from the bulk density. These 
eigenlevels often show collective behaviour, therefore, re- 
ferred to as collective modes. A detail discussion of the 
spectral properties of CWE can be found in [37( ■ 



IV. ESTIMATION OF THE MOMENTS OF 

EMERGING SPECTRA IN LINEAR RESPONSE 

REGIME 

We consider the power map for the correlation matrices 
which correspond to the WOE. As mentioned above, we 
use a power map in order to break the factorisable struc- 
ture of C while it is sufficient to consider q very close to 
1. We may expand the power map, for small a — (q— 1), 



C& = Cg>e*p fln[(CJ°V 



jk 



= C 3k + -C ]k \n{C%)[l + 0{a)], (10) 



where in the second equality we drop the superscript, 
using C for C^ -* onwards. Next we expand the eigenlevels 
Xj{a) of C("), as 



X j (a) = X j (0) + a(SX j )[l + O(a)}. 



(11) 



where the Xj (0)'s are the eigenlevels of the covariance ma- 
trix C, for j = 1, ..., N and the a(SXj)'s are the leading 
order corrections coming from the power mapping. For 
short time horizon, i.e. T < N, Xj(0) = for j < N- T. 
For small a, we assume that the statistics of the rela- 
tive changes in the eigenlevels is dominated by the lin- 
ear term. Bearing this assumption in mind, analytically, 
we derive estimates for the moments of the a(SXj)'s, i.e. 
in other word, in the linear response regime. We refer 
to a(SXj) as the eigenlevels of the emerging spectra, for 
j < N — T, otherwise as non-zero eigenlevel corrections. 
The term emerging spectra should not be confused with 
the small eigenlevels of C for n — 1 . As a matter of fact 
there are no such emerging spectra for k > 1. Without 
loss of generality we consider a > 0. 

Use of the perturbation theory for the individual eigen- 
levels is far too complicated in our case. Since we are 
mainly concerned about spectral statistics, it is sufficient 
to work in the matrix element space. For integer n > 0, 
we define the moments as 



i i N 

m n = _Tr[C(«)]" = -^(A,(a))«, 



3=1 



1 1 N 

^n = ^Tr [C]" = ^E^ ))"' (12) 



.7=1 



where the superscript {mp} denotes the moments which 
are closely described by the Marcenko Pastur density ([6]) . 
Next, for the correction eigenlevels we define the mo- 
ments, as 



Sm n 



i=i 



(13) 



In linear response regime Sm n may also be estimated by 

1 



5m n 



a term m — 



Tr (0))™-Ir (C) r 



(14) 



(0) 



Defining 5m n as moments of emerging spectra and 
Srrin P as the moments of non-zero eigenlevel corrections, 
we write 



8m n — Srrin 



orrin 



(15) 



Note that dm™ p = Sm n for T > N. 

It turns out that in order to calculate the averages in 
(Ti"4l one needs the moments of Cjk- Using the jpd of 
Ajk, given in (J3J, we calculate, 



(C jk ) 2n 



Ojk 



T 



+ (i-s jk ) 



ln T(2n + T/2) 
r(T/2) 
r(2n + l) T{T/2 + n) 
T 2n T(n + 1) T(T/2) 



( C rf"«=^(i) 2 " +,Q? ^p, (16) 



for integer n > 0. For completeness, we have included 
details of this calculation in the Appendix [Bj Using now 
Eq. ([M)) and Eq. (fl6|) . we calculate the first moment of 
the correction eigenlevels, as 



8m i 



a 

27V 



JV 



x;c«i°g[(ck) 5 






n\r / ^ ju yyjj) 



2N ^ dh 

3=1 



h=0 



(17) 



In the second equality of the above equation we use Eq. 
(fT6| for integer h > 1 and then we take the derivative for 
continuous h. From Eq. ()17|) we obtain 



<5toi = a 



], >" ( y 



*(l + T/2) 



(18) 



where ^(ir) is the digamma function. Similarly for the 
second moment we obtain 



Smo 




log- + ^(2 + T/2) 




where \t'(a;) = d${x)/dx. Up to the leading order, the 
large T asymptotic of (Smi) and (^7712) may be evaluated 
as 

a 



Smi ~ -, 



Smo 



a 

4k 



[log(T)+ Cl ] 2 + C2 ) 



(20) 



where c\ = 7 + log(2) - 2 = -0.729637... and c 2 = 
tt 2 /2 - 4 = 0.934802..., for 7 being the Euler constant. 
Note that the density defined by 5m n is doubly peaked 
for T < N, one which corresponds to the eigenlevels of 
emerging spectra and the other corresponds to the non- 
zero eigenlevel corrections. The two peaks converge to 
one as soon as T > N. 

As we know from the Marcenko Pastur result that the 
moments of the density depend on k alone. While mo- 
ments defined in (J15I) may have dependency on T also, 
as we show below, although to estimate either of the mo- 
ments is in no way straight forward. However, in the 
linear response regime we may assume that the ensemble 
averaged density of the non-zero correction eigenlevels 
could be approximated by a rescaled Marcenko Pastur 
density. Since non-negativity of eigenlevels is no longer a 
valid criterion, we allow the scale to assume negative val- 
ues as well. We note that this density should normalized 
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FIG. 3: Comparison of theory, shown by solid line, with sim- 
ulations, shown by symbols where T is the variable. In Figs. 
EJa) and EJb) we compare first two moments 5m\ and 8m2 
obtained from the numerical simulations with the theory (|20|) . 
In Figs. [3jc) and|3jd) we compare numerical results of scal- 
ing and shifting parameters s and r, obtained from Eqs. (|24l 
I25[) with theory (|26[) . In Figs. Efe) and[3jf) we compare nu- 
merical results of 5m™ p and (5m™ p with theory (|27[) . Finally, 



in Figs. [3jg) and[3jh) we compare 8ml an( A 5m 2 with the 
theory 1251 



to 1 for k > 1 and k otherwise. Besides, it is suggestive 
to use a shifting parameter. Finally, we make and ansatz 
for this density, 



Pi(SX) 



' 2tt(6X - r)s ' 



(21) 



Here S\ T = s(n~ 1 / 2 ±l) + r while r and s are the shifting 
and scaling parameters respectively. We fix these param- 
eters by calculating the first two moments of the density. 
For the above density (J21I) . we obtain 




for k > 1, 
for k < 1. 



(22) 



Similarly for the second moment we obtain 

1 + - I s 2 - r 2 + 2r(Smi), for k > 1, 



c nip I V K 

dm 9 F = < > 1 



1-1 — ) s — r K + 2r(Sm 1 p ), for k < 1. 



Inverting above relations, we write 



(23) 



<5m2 — <5mi 



k, for k > 1, 



— \ 8m™' p — , for k < 1, 



and 



5m\ — s, for k > 1, 

Jm™ 13 — s 



for k < 1. 



(25) 



We remark that the scaling parameter s should be a neg- 
ative quantity as the power mapping, for a > 0, shrinks 
the density of the Xj(a)'s for N - T < j < N. For k > 1 
the two parameters are known in terms of Smi and bm,2 
where both moments behave smoothly at k = 1. On the 
other hand, one a expects non-smooth behaviour in Sm™ p 
and Srn^ p at k = 1 , as the moments of the emerging spec- 
tra should not behave smoothly at k = 1. We now use 
results ([2U)l and (pMl [23]) to write asymptotic form of s 
and r for k > 1. For large T, we obtain AMndependent 
behaviour for both the quantities, e.g., 



--V>g(T)+ Cl p+ C2 , 



a 



V[iog(r) + ci] 



f'2 



r 



(26) 



<5m„ = Eli( AA i) n ]/ iV where aa j = A ! Q) - A i- Si 



,(o) 



larly, for Srrin' and <5m™ p we use AAj for < j < N — T 
and N ~T < j < N, respectively. As shown in the graph 
that our theory gives reasonable account for the numer- 
ical results. The non-smooth behaviour in the moments 
of non-zero or emerging spectra are well described by our 
theory. As we see in Figs. [H^c) and|3|d) that the scaling 
and the shifting parameters are independent of N. For 
the first moment we see some interesting features in Figs 
IHe) andlSJg), viz. (i) the two curves are almost sym- 
V^J metric about zero, iii) while 5m™ p reaches to minimum 

roughly when T < Nk/4, 8m\ reaches to the maximum 
value. The second moment for emerging spectra, on the 
other hand, increases for small T. This feature, however, 
can not be captured by our theory as k is small. 



V. POWER MAPPING FOR THE 
CORRELATED WISHART ENSEMBLES 

The CWEs are difficult to handle analytically in gen- 
eral and we mainly analyze the spectra numerically. We 
consider basically two types of ensemble or true corre- 
lation matrix £. In the first example, £ has £ block di- 
agonal structures, where Ni being the size of the i'th 
block such that ^2 t Ni — N while for the i'th block 
£jk = fijk + (1 — 8jk)ci- Matrix elements of the con- 
necting off-diagonal blocks are 0. Such examples quali- 
tatively correspond to the 'factor model' in quantitative 
finance [431, [44J . In the second example, £ has smooth 
band structure, e.g., Jyj. = c' :j ~ k ' which may correspond 
to situations where we expect that spatial correlations 
fall-off exponentially with the spatial distances. Without 
loss of generality in both cases we consider c > 0. 



Exploiting the A^-independence in these results, we may 
extrapolate them for n < 1. Then the estimation of 8m™ v 
and (5m" lp becomes straight forward, e.g., 



Sm 1 — ndrni + s(l — k), 



Sm™ p 



K5m 2 -KSm 1 + - ^—!—. (27) 



Using (|15l) . estimation of the moments of the emerging 
spectra become trivial. For large T and k < 1, one gets 



Sm\ 



(0) 



5m 



(0) 



-8(1- K) 

s 2 (1-k). 



(28) 



We mention that for small values of K, discrepancy in 
our approach becomes effective and such linear response 
theory fails to work. 

In Fig. [3] we have compared our theoretical esti- 
mates with numerical results for fixed A" = 512 and 
N = 1024. For the numerical results we have use 



A. Ensemble correlation matrix with block 
structures 

The simplest example for which we can readily obtain 
analytical results is t = 1. In this example £ is a dense 
matrix and its eigenlevels £, are simply given by £,■ = 1 — c 



for j = l,...,Ar-land£jv = ATc+1- 
Eqs. dll[9]) yields the density Hi 



■c. For this spectrum, 



p(A) = p>(\) + 8[\- 



(Nc + 1 - c)(Nck + 1 - c) 



Nck 



p'(X) = =K 



V(A + -A)(A-A_) 
2tt(1-c)A 



(29) 



where A± = (1 — c)(k -1 / 2 ± l) 2 . Note the similarity of 
p'(A) with the Marcenko-Pastur law ©, where only a fac- 
tor 1 — c appears extra in the place of a 2 . The delta func- 
tion appears in the above result as long as c > (A \/k) _1 . 
It is known from the work of Baik el al that the den- 
sity of the separated eigenlevels or the collective mode 
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FIG. 4: Results for power mapping of correlation matrices 
corresponding to CWE. We choose the true correlation ma- 
trix model where the matrix elements are £jk — 8jk+C (l — 8jk) 
and c = 0.5. Bulk densities of p(A), p 1 {8X m ) and p (<5A (0) ) 
are shown with their tail in the insets, respectively in Figs. 
IH a )i Hlb) and|4jc). Theory for the distribution of the iso- 
lated eigenlevels which is a Gaussian distribution as shown 
in Fig. 13 a), is obtained by using results for the mean and 
the variance given in [37j. We also remark remark about the 
existence of collective mode in Fig. HJc). Distribution of iso- 
lated eigenlevels in insets of Figs. |4jb) and|4jc) both are not 
Gaussian. Averaged mean positions as a function of the corre- 
lation coefficient c, for k = 1/2, are shown in Figs. Hfd), Sfe) 
and|4jf), respectively for the separated eigenlevels in p(A), 
p^SX^) andp (5A (0) ) 



is described by a Gaussian distribution [50|. Analytical 
results for the ensemble averaged mean and the variance 
are known [33] in terms of the spectra of £. 

To obtain some of the analytical results for the power 
mapped correlation matrices of this model we simply use 
the fact that for the bulk only the variance is reduced 
and derive results for the moments of bulk densities only. 
Using 1 — c as the variance in Eqs. (|B3[ IB51 IB6|) and 
following the calculations of Sec. Ill, we readily find 



5mi 



a(l-c)m(l-c), 

j— 2 a 2 (l-c) 2 



4k 



('2 



[ln(T) + ci - 21n(l - c)] 2 



(30) 



Next, extending the ansatz (f2"Tj) for the density p'(X), we 
calculate 



dm) 




nr, 



for k > 1, 
for k < 1. 



(31) 
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FIG. 5: Comparison of theory with numerical results where 
for the matrix elements of the true correlation matrix are 
£jfc = Sjk + c(l — S jk). We choose c = 0.5. First and second 
moments, Smi and Srri2 are shown respectively in Figs. [5ja) 
and^b). Scaling and shifting parameters s and r are shown 
respectively in Fig. [5jc) and Fig. [5jd). Moments of the den- 
sity of non-zero eigenlevel corrections, viz. Smi and Sm 2 , 
are shown respectively in Figs. [SJe) and[SJf). Moments of 

the emerging spectra, viz. Sm 1 and Sm 2 are shown respec- 
tively in Figs. EJg) and[5^h). Solid lines are theories and open 
circles represent numerical results. These moments are calcu- 
lated only for the bulk densities. Dashed lines are theoretical 
results of the WE. 



Sm 



(l) _ I m 2 s" 



2srmi, for k > 1, 



2 ~ W 



rri2S + r k + 2srm\ for k < 1, 



(32) 



where rni and mi are the first and the second moments 
p'(X) which in this case are 1 — c and (1 — c) 2 (l + 1/k), 
respectively and we have replaced the superscript {mp} 
by (1). For k > 1 and for a given c, s and t can be quickly 
estimated in terms of Smi and Sm2 ■ 



n[Sm2 — Sirij] 

S = J (1-c) 2 ' 

r = 5m\ — s(l — c). (33) 

Since this result confirms possibility of extrapolating s 
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FIG. 6: Non-zero eigenlevel density and emerging spectra 
eigenlevel density, of O 9 ' where the true correlation matrix £ 
has ^-diagonal blocks and i = 3, respectively in Figs. [6ja) and 
Ob). We have considered JVi = N/2, N 2 = N 3 = N/4 while 
correlation coefficients are respectively Ci = 0.9, C2 = 0.45 
and C3 = 0.225. In the inset of Fig. EJa) density of the iso- 
late eigenlevels are shown with the theory. First and second 

moments, Srrii , Sm 2 , of po(/V ') are shown as function of 
time horizon T, respectively in Fig. [BJc) and in Fig. H^d). In 
Fig. [6ja) solid lines represent theory obtained from the nu- 
merical solution of Eq. © and symbols represent numerical 
data. For c = 0, which corresponds to WE, only theories are 
shown by dashed line. 



and r for k < 1, therefore for the moment 8ml we get 



,(!) 



= kStui + (1 — c)s(l — k), 



(34) 



while the result for second moment is the same as ob- 
tained in Eq. ((27)) . 

In Fig. Ufa), Fig. gjb) and Fig. gjc), we show bulk 
densities for the non-zero eigenlevels, non-zero correction 
eigenlevels and emerging spectra of C^ q ' , respectively, for 
c = 0.5. Theory for Fig. QJa) comes from Eq. (|29|) 
while in Fig. |U[b) it comes from the ansatz for p' 1 (5A) 
with s and r extrapolated for n < 1. Theory of Fig. 
SJc) is not known. Densities observed in these figures 
are different from those for the WE. In the inset of these 
figures we show the density of isolated eigenlevels which 
is remarkable in Fig. BJc). In Fig. 0|d), Fig. [Ife) and 
Fig. B|f), we show averaged mean of the largest eigenlevel 
respectively for p(X), p^SX^) and p (SX^). If the the 
separation is large then the corrections due to the power 
map to the averaged mean of the largest eigenlevel may 
be estimated by 5Xn ~ aAjvlog(A^r)/2. In Fig. |4je) 
we have compared this theory with numerics. Note that 
the collective mode that appeared in Fig. 2]Jb) is in the 
other side of the peak of the density in bulk, as opposed 
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FIG. 7: Eigenvalue density, p(X), of C, emerging spectral den- 
sity, po(^ )> °f C (9) for different values of the correlation 
coefficient c are shown in Fig. [TJa) and in Fig. UJb), respec- 
tively. First and second moments, 5m 1 , Sm 2 , of /^(A' ') 
are shown as function of time horizon T, respectively in Fig. 
[3c) and in Fig. [TJd). In Fig. [TJa) solid lines represent theory 
obtained from the numerical solution of Eq. ((9j and symbols 
represent numerical data. For c = 0, which corresponds to 
WE, only theories are shown by dashed line. 



to what is seen in Fig. |4ja) and in Fig. QJc). 

In Fig. [5] we compare the numerical moments with 
theoretical results. We also show in this figure theoretical 
results of the WE, those we derived in Sec. IV. The 
moments of pto)(SX^) are observed to be smaller than 
their WE counterparts. Qualitatively similar results are 
obtained for t — 3 where we have considered N\ = N/2, 
N2 = N3 = N/4 with correlation coefficients respectively 
as Cl = 0.9, c 2 = 0.45 and c 3 = 0.225. Here in Fig. E^b) 
we notice a hump in the tail, indicating that the delta 
functions are about to merge with the bulk. 



B. Ensemble correlation matrix with smooth band 
structure 

In our second example, i.e. ^ = a-' ', we present 
numerical results only. The density of eigenlevels of the 
correlation matrices is obtained by solving the Eq. Q 
numerically for the spectrum of £ while k = 1/2. For dif- 
ferent values of correlation coefficient, viz. c = 0.8, 0.5 
and c = 0, densities are compared in Fig. [7] (a) where we 
find a good agreement of the theory with numerics. As 
shown in this figure, majority of eigenlevels move towards 
zero with c while large eigenlevels moves away from the 
upper edge of Marcenko-Pastur density. In Fig. [7J (b) 
we compare densities of emerging spectra which change 



with c as well. We remark that in this figure the upper 
and the lower both spectral edges moves efficiently with 
c because there is no hard edge as such there is in the 
density of emerging spectra. In Figs. [7J (c) and [7] (d) the 
first two moments of JJ^X^) are compared for different 
time horizon. These moments qualitatively explain what 
we see in Fig. 0(b). Interestingly for strong correlations 
these moments develop convex profile which worths fur- 
ther investigations. 



VI. CONCLUSION 



it would be worth investigating properties of eigenvectors 
corresponding to the emerging spectra, as well. 
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In conclusion, we have studied spectral behaviour of 
random correlation matrices estimated for short time 
horizons and deformed slightly by the power map. This 
study is relevant in the multivariate analysis of large com- 
plex systems. In the context of financial time series we 
have shown the importance of the power map with an 
illustration of this method for singular correlation matri- 
ces. We have mainly concentrated on the statistics of the 
spectra emerging due to the exponent of the power map 
being very close to one. On one hand this is interesting 
due to the spontaneous breaking of the degeneracy at 
zero while on the other hand we get a clearer picture for 
the spectral statistics and analytical treatment becomes 
easier. Our analytic results for the Wishart ensembles set 
a benchmark which can be important in detecting the 
true correlations. In the derivation of these results we 
have introduced scaling and shifting parameters which 
are calculated for k > 1. Then by extrapolating these 
parameters for k < 1 we have calculated estimates of 
the first and the second moments of the emerging spec- 
tra. These results are valid in a linear response regime 
and therefore are not valid for very small K, at least for 
the higher order moments. We should emphasized that 
smaller k values are also important for the real data. 
Moreover, for a very singular matrix statistical analy- 
sis for the emerging spectra suits better than the regu- 
lar nonzero spectra which has only few eigenlevels. It 
is therefore important to obtain results for the spectral 
statistics for very small k values. 

We have also considered true correlations in the matrix 
elements. We have focused on mainly two types exam- 
ples, viz. when the ensemble correlation matrix has a 
block diagonal structure and when it has smooth band 
structure. For the simplest case of the former model we 
have been able to obtain theoretical results for the bulk. 
In that case appearance of collective mode in the density 
of the emerging spectra is interesting, although analyt- 
ically it can not be described by using linear response 
theory. In the second case we have numerically shown 
that how true correlations change the moments appre- 
ciably. With both examples we have established the that 
the emerging spectra is also very sensitive to correlations. 
As far as analytic treatment in concerned we believe that 
some scaling and shifting parameter may be useful in ex- 
plaining these numerical results. Finally we mention that 



Appendix A: Details of the portfolio optimization 
study 

The weight vector of the minimal variance portfolio is 
calculated as 



"opt 



e«£- 



(Al) 



where e is a vector of length N with all entries set to one, 
e' denotes the transposed vector. In order to calculate 
the optimal weights (|A1|) . we need to know the covari- 
ance matrix £ of the iV stock returns. In practice, this 
covariance matrix has to be estimated using historical 
data. Here we consider a model setting with N = 100 
stocks, a model correlation matrix Co with 5 blocks of 
size 20, corresponding to industry sectors, and randomly 
distributed but fixed standard deviations a^. In a factor 
model that reflects our correlation matrix Co, we sim- 
ulate time series of length TV, estimate the correlation 
matrix (7( sam P) anc [ apply the power map to arrive at the 
matrix C^ q > with entries 



r (q) _ ■ r (samp) 



c 



(samp) I <? 

kl " I 



(A2) 



We multiply with the standard deviations to get the ele- 
ments of the covariance matrix, 

£w = witift . (A3) 

With this covariance matrix we calculate the weights 



"opt 



e«£- 



(A4) 



Using the model correlation matrix Co and the cor- 
responding covariance matrix So = crCoc, with a = 
diag(ai, . . . , <7jv), the actual portfolio variance for the 
weights (JA4I) reads 



fi2 = w* t S W op t , 



(A5) 



whereas the minimal variance S1q is obtained by calculat- 
ing the optimal weights (|A1|) for Sq. 
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Appendix B: Details of the derivation of Eq.()16p 

We begin with considering the following binary associ- 
ations, for integer n > 1, 

1 T 

(Cjk) n = — 2_^ Aj mi Ak mi \Cjk) 

mi — 1 ' ' 

In T 
T^ln / j / j Aj r ,i 1 Aj ir1 - ll ...Aj rrii ^femj ' l '-™-J7ri2n kni2n 

i=2 {m}=l l ■ 

In T 
rp9„ / j / j slj mi A kmi ...Aj rrii Akrni ■■■Aj m2n A) zm2ri . 



y2 



=2 {m}=l l 



(Bl) 



These binary associations results in terms of the moments 
of the diagonal elements 

x (Q fc ) 2n - 2 C fefe 



<** ( ' + ^V 2 ) E (^) 2 - 



T 



f=i 



r(2n + l)r(n + 2 - £) 
T(n + l)r(2n - 2^ + 3)(2T)^ 1 

r(2n + l) 



r(n + l)(2T)' 



(C fcfc ) n , 



(B2) 



where we have considered the mean and the variance of 
Ajk to be and 1, respectively. Moments for the diagonal 
terms are comparatively easier to obtain. Bearing similar 
binary associations in mind we readily write the result for 
integer n > with arbitrary variance a 2 of Aj k : 



Note here that the second equality in the above equation 
is the extension of the first equality for n — 0. Now we 
use the result (|B3|) in Eq. (|B2j) with a 2 = 1 and derive 
exact result for the even moments of the matrix elements 



(C jk ) 2n 



^ 2 2(n- 



r(2n+l) 



2» 



S jk T ( 1 



2n- 1 



ty 



T(n + l)T{T/2)T 

. a r(n + 2 - £)r(2n - t + T/2) 



T 



T(2n-2£ + 3) 



T(n + T/2) 
r(2n + l) 



<5 jfe T | 1 



2n-l 

t 



r(n + i)r(r/2)r 2 

r(l/2)F(2n + T/2) - r(n + l/2)r(n + T/2) 
r(n+l/2)(T + 2n- 1) 

+ T(n + T/2) 

r(2n + l) 



r(n+l)r(T/2)T 2 ™ 
+ T{n + T/2) (1 - S jk ) 



S jk - 



r(l/2)r(2n + T/2) 
r(n + 1/2) 



(B4) 



On further simplification it results in the first equality of 
Eq. ([To]) . For arbitrary variance we write Eq. (fTr?)) as 



(C jfc ) 2 " = ^ fe ( — 



2a 2 \ 2 "r(2n + T/2) 



T(T/2) 



+ (1 - S jk ) 



<t 2 \ r(2n + l)r(T/2 + n) 



T) r(n+l) T(T/2) 



(B5) 



Following the similar steps we obtain the second equality 
of Eq. (fTrJj) , which is with a 2 may be written as 



(c 33 Y = ]> 2 f 1 



2(^-1) 
T 



2a 2 V T(n + T/2) 



T / T(T/2) 



(B3) 



TO^-^I^)" 1 ^^ . (B6) 
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